Skip to content

OLS-3593 enable init container comparison for app server deployment - #1897

Merged
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
raptorsun:OLS-3593-fix-rag-init-container-reconciliation
Aug 5, 2026
Merged

OLS-3593 enable init container comparison for app server deployment#1897
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
raptorsun:OLS-3593-fix-rag-init-container-reconciliation

Conversation

@raptorsun

@raptorsun raptorsun commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Fix: detect changes to .spec.ols.rag (RAG image references) so they trigger a deployment rollout
  • Root cause: DeploymentSpecEqual was called with compareInitContainers=false, and no other change detector tracked RAG images — so RAG image changes were silently ignored

Why not just enable init container comparison?

The obvious fix — flipping compareInitContainers from false to true — causes an infinite reconciliation loop when RAG is configured.

When RAG images are present, the operator sets an image.openshift.io/triggers annotation on the deployment (rag.go:62-79). This tells OpenShift's image trigger controller to resolve ImageStreamTags and replace the init container image field with the resolved @sha256:... digest. With compareInitContainers=true, this creates a fight between two controllers:

  1. Operator reconciles → generates desired deployment with the user-specified image tag (e.g., my-image:latest)
  2. OpenShift trigger controller → resolves the tag and writes the SHA digest into the deployment (e.g., my-image@sha256:abc...)
  3. Operator reconciles againContainerSpecEqual compares a.Image == b.Image, sees tag != digest, detects a "change"
  4. Operator updates the deployment back to the tag → goto step 2

This was confirmed in e2e testing where the deployment generation reached 5663 vs observed generation 2303 (~5-6 updates/second).

Solution

Track a SHA-256 hash of the CR's .spec.ols.rag in a deployment annotation (ols.openshift.io/rag-spec-hash). When the user changes RAG image references, the hash changes and triggers a deployment rollout — without comparing init container images directly and without conflicting with OpenShift's image trigger mechanism.

Test plan

  • make test passes — all unit tests green
  • bundle-e2e-4-21 CI passed (including BYOK/RAG tests)
  • Deploy OLS with a RAG image configured, change the image tag, and verify the operator rolls out a new deployment

Fixes: OLS-3593

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@raptorsun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 44 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 4a731d03-43dc-4acc-a4a0-c37029a5718b

📥 Commits

Reviewing files that changed from the base of the PR and between d3e5e5a and 621853e.

📒 Files selected for processing (2)
  • internal/controller/appserver/deployment.go
  • internal/controller/utils/constants.go
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@openshift-ci
openshift-ci Bot requested review from bparees and xrajesh July 30, 2026 16:16
@raptorsun

Copy link
Copy Markdown
Contributor Author

/retest

@raptorsun
raptorsun force-pushed the OLS-3593-fix-rag-init-container-reconciliation branch from 56a1912 to 7eac125 Compare August 3, 2026 07:48
@openshift-ci openshift-ci Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 3, 2026
The app server deployment update check called DeploymentSpecEqual with
compareInitContainers=false, so changes to RAG images in .spec.ols.rag
were never detected. Simply enabling init container comparison causes
an infinite reconciliation loop because OpenShift's image trigger
controller resolves ImageStreamTags to SHA digests, which the operator
then overwrites with the original tag on the next reconcile.

Instead, track a SHA-256 hash of the RAG spec in a deployment
annotation. When the user changes .spec.ols.rag, the hash changes and
triggers a deployment rollout without conflicting with the image
trigger mechanism.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@raptorsun
raptorsun force-pushed the OLS-3593-fix-rag-init-container-reconciliation branch from 7eac125 to 621853e Compare August 3, 2026 08:03
@openshift-ci openshift-ci Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 3, 2026
@raptorsun

Copy link
Copy Markdown
Contributor Author

/approve

@openshift-ci

openshift-ci Bot commented Aug 3, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: raptorsun

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 3, 2026

@blublinsky blublinsky left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: OLS-3593 RAG spec hash detection

Nice, clean fix for a subtle bug. The hash-based approach avoids the image trigger controller fight elegantly.

Non-blocking observations

  1. json.Marshal error discarded (deployment.go:ragSpecHash()) — data, _ := json.Marshal(...) drops the error. In practice RAGSpec is a simple struct and won't fail, but a //nolint:errcheck comment or a brief rationale would help future readers. Not blocking since sha256.Sum256(nil) is deterministic (no spurious rollouts).

  2. No unit test for the new logic — a table-driven test for ragSpecHash() (empty / single / multiple entries) and one case in updateOLSDeployment verifying the annotation-based rollout would be straightforward additions. Not blocking since e2e coverage exists.

Both are nice-to-haves. LGTM otherwise — the approach is sound and the PR description is excellent.

@blublinsky

Copy link
Copy Markdown
Contributor

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Aug 4, 2026
@raptorsun

Copy link
Copy Markdown
Contributor Author

/test bundle-e2e-4-21

@raptorsun

Copy link
Copy Markdown
Contributor Author

/test bundle-e2e-4-21
CI could not claim a cluster

@openshift-ci

openshift-ci Bot commented Aug 5, 2026

Copy link
Copy Markdown

@raptorsun: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot
openshift-merge-bot Bot merged commit 10e8309 into openshift:main Aug 5, 2026
13 checks passed
@raptorsun
raptorsun deleted the OLS-3593-fix-rag-init-container-reconciliation branch August 5, 2026 12:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants